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GRADED ZOOMING 
[0001] The United States Government has rights in this invention pursuant to 
Contract No. W-7405-ENG-48 between the United States Department of Energy and 
the University of California for the operation of Lawrence Livermore National 
Laboratory. 

BACKGROUND 

Field of Endeavor 

[0002] The present invention relates to image enhancement and more 
particularly to graded zooming. 

State of Technology 
[0003] U. S. Patent No. 6,201,574 to H. Lee Martin, patented March 13, 2001, 
shows a motionless camera orientation system distortion correcting sensing 
element. Camera viewing systems are utilized in abundance for surveillance, 
inspection, security, and remote sensing. Remote viewing is critical for robotic 
manipulation tasks. Close viewing is necessary for detailed manipulation tasks 
while wide-angle viewing aids positioning of the robotic system to avoid 
collisions with the workspace. The majority of these systems use either a fixed- 
mount camera with a limited viewing field, or they utilize mechanical pan-and- 
tilt platforms and mechanized zoom lenses to orient the camera and magnify its 
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image. In the applications where orientation of the camera and magnification of 
its image are required, the mechanical solution is large and can subtend a 
significant volume making the viewing system difficult to conceal or use in close 
quarters. Several cameras are usually necessary to provide wide- angle viewing 
of the workspace. In order to provide a maximum amount of viewing coverage 
or subtended angle, mechanical pan/tilt mechanisms usually use motorized 
drives and gear mechanisms to manipulate the vertical and horizontal 
orientation. An example of such a device is shown in U.S. Pat. No. 4,728,839 
issued to J. B. Coughlan, et al., on Mar. 1, 1988. Collisions with the working 
environment caused by these mechanical pan/tilt orientation mechanisms can 
damage both the camera and the worksite and impede the remote handling 
operation. Simultaneously, viewing in said remote environments is extremely 
important to the performance of in-on and manipulation activities. Camera 
viewing systems that use internal optics to provide wide viewing angles have 
also been developed in order to minimize the size and volume of the camera and 
the intrusion into the viewing area. These systems rely on the movement of 
either a mirror or prism to change the tilt-angle of orientation and provide 
mechanical rotation of the entire camera to change the pitch angle of orientation. 
Using this means, the size of the camera orientation system can be minimized, 
but "blind spots" in the center of the view result Also, these systems typically 
have no means of magnifying the image and or producing multiple images from 
a single camera. 



Attorney Docket No. IL-10830 
Express Mail Certificate No. EL534392888US 

[0004] U. S. Patent No. 6,320,979 to Roger D. Melen, patented November 20, 
2001 shows depth of field enhancement. Depth of field is a measurement of the 
range of depth along a view axis corresponding to the in-focus portion of a three 
dimensional scene being imaged to an image plane by a lens system. Several 
parameters of a lens system influence the depth of field of that lens system. In 
general, optical systems with high magnification, such as microscopes, have 
small depths of field. Also, optical systems which use large aperture lens systems 
to capture more light generally have small depths of field. In some situations it is 
desirable to have the benefits of a larger depth of field without giving up those 
optical qualities which generally result in small depths of field. For example, 
some analyses of microscopic specimens would be aided by the availability of a 
high magnification microscope with a relatively large depth of field. Such a 
microscope could be used to more clearly image the full structure of a 
microscopic object which is three dimensional in nature. Ordinary microscopes 
generally allow the clear viewing of a thin section of such a three dimensional 
specimen, due to the small depth of field of those microscopes. Portions of the 
specimen which are on either side of the in- focus section will be out of focus, 
and will appear blurry. The ability to clearly see the full three dimensional 
structure of a specimen would aid in the understanding of the structure of that 
specimen. This would be especially useful when used in conjunction with 
biojective microscopes which allow a user to view a specimen stereoscopically. 
Another situation in which a small depth of field can pose problems is the low 
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light photography of a scene with large depth variations. An example of this is a 
landscape scene including foreground objects photographed at night. In order to 
get sufficient light onto the film at the image plane of the camera, a large 
aperture lens must generally be used. A large aperture lens, however, will result 
in a relatively small depth of field. Because of the small depth of field, only a 
portion of the scene being photographed will be in focus. A conventional method 
of imaging the depth information of a three dimensional microscopic scene is 
confocal microscopy. In confocal microscopy a single photodetector is situated 
behind a pinhole in an opaque screen. An objective lens focuses light from an 
illuminated point onto the pinhole, and the screen masks out any non-focused 
light. The illuminated point is generally illuminated by an intense, focused light 
source, such as a laser. The illuminating light source and the pinhole must be 
scanned over a microscopic specimen, either one point at a time or in a series of 
lines, in order to build up information for the whole region of interest. Depth 
information can be extracted from the data recorded by the photodetector. The 
information obtained from a confocal microscope can be used to image the three 
dimensional structure of microscopic specimens, but such a system is too 
complex and expensive for typical microscopy. Also, confocal microscopy is 
limited to situations in which microscopic specimens are being imaged, and is 
not practical for imaging macroscopic scenes. What is needed is a system capable 
of producing an image of a three dimensional scene with enhanced focus over a 
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large depth of field, without sacrificing optical qualities which ordinarily require 
a small depth of field. 

[0005] U. S. Patent No. 6,332,044 to Robert P. Loce and Michael Branciforte, 
patented December 18, 2001, shows a system and method for enhancement of 
image contour fidelity. The invention relates generally to hierarchically 
organized filters for processing digital images, and more particularly to the use 
of hierarchically organized template-matching filters to accomplish the 
resolution enhancement in a cost and computationally efficient manner. 

SUMMARY 

[0006] Features and advantages of the present invention will become 
apparent from the following description. Applicants are providing this 
description, which includes drawings and examples of specific embodiments, to 
give a broad representation of the invention. Various changes and modifications 
within the spirit and scope of the invention will become apparent to those skilled 
in the art from this description and by practice of the invention. The scope of the 
invention is not intended to be limited to the particular forms disclosed and the 
invention covers all modifications, equivalents, and alternatives falling within 
the spirit and scope of the invention as defined by the claims. 
[0007] The present invention provides a system for balancing resolution of a 
scene having a bottom and a top and a near field and a far field. The scene is 
represented by a plurality of lines of pixels that capture the scene's near field and 
maintains resolution in the scene's far field. A specific zooming scale factor is 
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applied to each of the lines of pixels. The scale factor is increased from the bottom to 
the top to capture the scene in the near field, yet maintain resolution in the scene in 
the far field. 

[0008] The invention is susceptible to modifications and alternative forms. 
Specific embodiments are shown by way of example. It is to be understood that 
the invention is not limited to the particular forms disclosed. The invention 
covers all modifications, equivalents, and alternatives falling within the spirit 
and scope of the invention as defined by the claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0009] The accompanying drawings, which are incorporated into and 
constitute a part of the specification, illustrate specific embodiments of the 
invention and, together with the general description of the invention given 
above, and the detailed description of the specific embodiments, serve to explain 
the principles of the invention. 

FIG. 1 illustrates an embodiment of the present invention. 

FIG. 2 shows a scene in the normal view. 

FIG. 3 shows the scene shown in FIG. 2 as seen through graded zooming. 

FIG. 4 illustrates how continuously increasing a scale factor from bottom 
to top can isolate a set of pixels. 

FIG. 5 illustrates that if the set of pixels shown in FIG. 4 are removed, a 
new pixel map is produced representing graded zooming. 
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FIG. 6 is a flow chart that illustrates another embodiment of the present 
invention. 

DETAILED DESCRIPTION OF THE INVENTION 
[0010] Referring now to the drawings, to the following detailed information, 
and to incorporated materials; a detailed description of the invention, including 
specific embodiments, is presented. The detailed description serves to explain the 
principles of the invention. The invention is susceptible to modifications and 
alternative forms. The invention is not limited to the particular forms disclosed. 
The invention covers all modifications, equivalents, and alternatives falling 
within the spirit and scope of the invention as defined by the claims. 
[0011] An embodiment of a system constructed in accordance with the 
present invention is illustrated in FIG. 1. The system is designated generally by 
the reference numeral 10. The system 10 is a surveillance system that includes a 
surveillance camera 11 connected to a digital network 13. The camera 11 is a 
digital camera interconnected via the digital network 13. The digital network 13 
includes or is connected to a computer 12. The computer 12 is a device capable of 
performing a series of arithmetic or logical operations. Computer may be general 
purpose computer or a microprocessor that forms a part of the overall system. 
The computer 12 is a general purpose computer. 

[0012] Security surveillance systems usually provide views over well-defined 
areas. The surveillance activity outside of the defined security area is of little 
interest, while the activity inside the defined area can be of critical importance. 



Attorney Docket No. IL-10830 
Express Mail Certificate No. EL-534392888US 

Unfortunately, camera views are rarely a good fit for covering the denned area. 
When, the cameras don't fit the defined area, the solution is usually to add more 
cameras. This can be extremely expensive, because it usually means adding 
infrastructure such as cable plant, poles, distribution networks, and the cameras 
themselves. 

[0013] Surveillance cameras are most often mounted such that the view 
"looks" across the area (parallel to the ground) rather than directly down on the 
area. With cameras looking across an area, the objects in the near field naturally 
appear larger than objects in the far field. As such, a typical camera view, as seen 
on a video monitor, has the closest view and best resolution at the bottom and 
the farthest view, and lowest resolution at the top. 

[0014] Cameras must be "zoomed out" in order to ensure that important 
items in the near field are captured. This results in views in the far field being 
much wider than desirable and causes the resolution in the far field to be much 
lower than desirable. Cameras are usually adjusted such that the coverage in 
the near field is complete, and such that the resolution in the far field is 
sufficient for an assessment. When these two opposing criteria cannot be met, 
additional cameras are required. 

[0015] The system 10 provides balances the resolution within a video scene or 
still image, thereby allowing the camera 11 to capture the full scene in the near 
field, yet maintain resolution in the far field. While this may be possible through 
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specially built lenses, the system 10 utilizes digital images captured through 
conventional lenses and can be accomplished at a very low cost. 
[0016] Using digital frames, the system 10 applies a specific zooming scale 
factor to each horizontal line of pixels. The system 10 uses graded zooming, 
continuously increasing the scale factor from bottom to top, causing figures in 
the far field to appear much closer. The large pixel arrays currently available 
through digital photography and video make over-sampling possible with no 
visible scene degradation. 

[0017] The number of pixels used in each horizontal line of pixels is constant, 
however the rate of over-sampling of pixels is reduced from bottom to top 
according to a scale factor. For instance, if the view at the top of a sample 
picture is zoomed to 2X of that of the bottom of the sample picture, the bottom 
line is over-sampled at a rate of 2, while the top line is not over-sampled at all. 
[0018] In system 10, each successive line from bottom to top is over-sampled 
by the following equation: 

S = l + (Zt-Zb)(N-n)/N 

Where S is the rate of over-sampling, N is the number of horizontal lines of 
pixels, n is the horizontal line of pixels number counting from bottom to top, Zt 
is the zoom ratio at the top of the sample picture, and Zb is the zoom ratio at the 
bottom of the sample picture. 
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[0019] The system 10 is illustrated utilizing zooming in one dimension only. 
It is to be understood that zooming in other dimensions is available and is 
included in other embodiments. Referring again to FIG. 1, the figure will be 
used to describe another embodiment of a system constructed in accordance 
with the present invention. Pictures are two-dimensional and simply applying 
Equation 1 to the horizontal lines would result in images that appear stretched 
horizontally. In this embodiment a similar equation is applied in the vertical 
dimension resulting in images with correct aspect ratios. Each successive line 
from side to side is over-sampled by the following equation: 

S = l + (Zt-Zb)(N-n)/N 
[0020] Where S is the rate of over-sampling, N is the number of vertical lines 
of pixels, n is the vertical line of pixels number counting from one side to the 
other side, Zt is the zoom ratio at the one side of the sample picture, and Zb is 
the zoom ratio at the other side of the sample picture. Applying Equation 1 in 
the vertical direction results in skipping lines. 

[0021] Referring now to FIGS. 2 and 3, another embodiment of a system 
utilizing the present invention is illustrated. The scene in FIG. 2 is generally 
designated by the reference numeral 20. The scene in FIG. 3 is generally 
designated by the reference numeral 30. The system of the embodiment 
illustrated by FIGS. 2 and 3 provides the ability to increase the resolution in the 
far field resolution of video or still frame images, while maintaining full coverage 
in the near field. 

-10- 
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[0022] A typical camera view, as seen on a video monitor, has the closest view 
and best resolution at the bottom and the farthest view, and lowest resolution at 
the top. Cameras must be /y zoomed out" in order to ensure that important items 
in the near field are captured. This results in views in the far field being much 
wider than necessary and therefore causes the resolution in the far field to be 
much lower. Cameras are usually adjusted such that the coverage in the near 
field is complete, and such that the resolution in the far field is sufficient for an 
assessment. When these two opposing criteria cannot be met, additional cameras 
are required. 

[0023] The system of the embodiment of the present invention illustrated in 
FIGS. 2 and 3 balances the resolution within a video scene or still image, thereby 
allowing the camera to capture the full scene in the near field, yet maintain 
resolution in the far field. This system can be used in any situation where 
maximum resolution is required over a large viewing area, and the resulting 
picture distortion is acceptable. This system has its most effective 
implementation in digital image capturing. 

[0024] Referring again to FIGS. 2 and 3, the system provides a means of 
equalizing far field and near field resolution in video scenes and still images. 
This is done by providing a wide view angle in the near field and a narrow view 
angle in the far field, and by continuously reducing the view angle between the 
near field and the far field within a single frame. FIG. 2 shows a scene 20 in the 
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normal view. FIG. 3 shows a scene 30, which is the same as the scene 20; 
however the scene 30 is as seen through graded zooming. 

[0025] While this may be possible through specially built lenses, this 
embodiment of the present invention relies on digital images captured through 
conventional lenses and can be accomplished at a very low cost. Using digital 
frames, this embodiment of the present invention applies a specific zooming 
scale factor to each horizontal line of pixels. 

[0026] Two lions 21 and 22 are shown in FIG. 2. The lions 21 and 22 are 
actually the same size; however, as seen in the normal view of FIG. 2 the lion 22 
appears to be smaller. The camera taking the normal view of FIG. 2 is mounted 
such that the view // looks ,/ across the area, parallel to the ground. With the 
camera looking across an area, the lion 21 in the near field appears larger than 
the lion 22 in the far field. As such, a typical camera view, as seen on a video 
monitor, has the closest view and best resolution at the bottom and the farthest 
view, and lowest resolution at the top. 

[0027] The scene 30 in FIG. 3 show how continuously increasing the scale 
factor from bottom to top causes the lion 32 in the far field to appear much closer 
when using graded zooming. The large pixel arrays currently available through 
digital photography and video make over-sampling possible with no visible 
scene degradation. 

[0028] This embodiment of the present invention can be best demonstrated 
zooming in one dimension only. The number of pixels used in each horizontal 
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line of pixels is constant, however the rate of over-sampling of pixels is reduced 
from bottom to top according to the scale factor. For instance, if the view at the 
top of the picture is zoomed to 2X of that of the bottom of the picture, the bottom 
line is over-sampled at a rate of 2, while the top line is not over-sampled at all. In 
one possible implementation, each successive line from bottom to top is over- 
sampled by the following equation: 

S = l + (Zt-Zb)(N-n)/N 
Where S is the rate of over-sampling, N is the number of horizontal lines, n is the 
horizontal line number counting from bottom to top, Zt is the zoom ratio at the 
top of the picture, and Zb is the zoom ratio at the bottom of the picture. 
Pictures, however, are two-dimensional and simply applying Equation 1 to the 
horizontal lines would result in images that appear stretched horizontally. 
Applying a similar equation in the vertical dimension results images with correct 
aspect ratios. Applying Equation 1 in the vertical direction results in skipping 
lines. 

[0029] Referring now to FIGS. 4 and 5, another embodiment of the present 
invention is illustrated. A pixel map is show in FIG. 4 and generally designated 
by the reference numeral 40. The pixel map 40 includes the lighter colored pixels 
41 and the darker colored pixels 42. A pixel map is show in FIG. 5 and generally 
designated by the reference numeral 50. The pixel map 50 includes the lighter 
colored pixels 51 and the darker colored pixels 52. 
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[0030] The pixel maps 40 and 50, in FIGS. 4 and 5, explain how pixels might 
be sampled if they are over-sampled using graded zooming. The same number 
of pixels are present in both pixel map 40 and pixel map 50. FIG. 4 illustrates 
how continuously increasing a scale factor from bottom to top can isolate the set 
of darker colored pixels 42. If the lighter colored pixels 41 are then removed 
from the pixel map 40, leaving only the set of darker colored pixels 42, a new 
pixel map can be produced. The new pixel map is represented by the pixel map 
50 shown in FIG. 5. 

[0031] The pixel map 50 of FIG. 5 includes a scene 52 represented by the 
darker colored pixels 52. Graded zooming causes the top portion of the scene 
represented by pixels 52 in the far field to appear much closer. The number of 
pixels used in each horizontal line of pixels is constant, however the rate of over- 
sampling of pixels is reduced from bottom to top according to the scale factor. 
For instance, if the view at the top of the picture is zoomed to 2X of that of the 
bottom of the picture, the bottom line is over-sampled at a rate of 2, while the 
top line is not over-sampled at all. In one possible implementation, each 
successive line from bottom to top is over-sampled by the following equation: 

S = l + (Zt-ZbHN-n)/N 
Where S is the rate of over-sampling, N is the number of horizontal lines, n is the 
horizontal line number counting from bottom to top, Zt is the zoom ratio at the 
top of the picture, and Zb is the zoom ratio at the bottom of the picture. 
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[0032] Pictures, however, are two-dimensional and simply applying Equation 
1 to the horizontal lines would result in images that appear stretched 
horizontally. Applying a similar equation in the vertical dimension results 
images with correct aspect ratios. Applying Equation 1 in the vertical direction 
results in skipping lines. 

[0033] Another embodiment of a system constructed in accordance with the 
present invention is illustrated in FIG. 6. The system is designated generally by 
the reference numeral 60. The system 60 includes: a scene 61 having a bottom, a 
top, a near field, a far field, and a plurality horizontal lines of pixels that 
captures the scene's near field and maintains resolution in the scene's far field. 
The scene is captured by a camera 62. The system 60 includes a computer- 
readable medium 63. The computer-readable medium 63 includes a Windows 
NT 4.0 personal computer interconnected with a 100 Mbps local area network. A 
specific zooming scale factor 64 is applied to each of the horizontal line of pixels. 
The system operates by continuously increasing the scale factor of the horizontal 
line of pixels from the bottom to the top to capture the scene in the near field, yet 
maintaining resolution in the scene in the far field as illustrated by block 65. The 
pixels in the lines of pixels are over-sampled as illustrated by block 66. The lines 
of pixels are over-sampled using graded zooming in a horizontal and a vertical 
direction. The number of the pixels used in the horizontal line of the pixels is 
constant, however the rate of over-sampling of the pixels is reduced from the 
bottom to the top according to a scale factor wherein the horizontal line of the 
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pixels at the top of the scene is zoomed to 2X of that of the horizontal line of the 
pixels at the bottom of the scene and the bottom line is over-sampled at a rate of 
2, while the top line is not over-sampled at all. This is represented by block 67. 

[0034] A computer program utilizes equation 68 as follows: 

S = l + (Zt-Zb)(N-n)/N 
where S is the rate of the over-sampling, N is the number of the plurality 
horizontal lines, n is the horizontal line number counting from the bottom to the 
top, Zt is the zoom ratio at the top of the scene, and Zb is the zoom ratio at the 
bottom of the scene. 

[0035] The system 60 balances the resolution within a video scene or still 
image, thereby allowing the camera 62 to capture the full scene 61 in the near 
field, yet maintain resolution in the far field. Using digital frames, the system 60 
applies a specific zooming scale factor 64 to each horizontal line of pixels. The 
system 60 continuously increases the scale factor from bottom to top 65 causing 
figures in the far field to appear much closer when using graded zooming. The 
number of pixels used in each horizontal line of pixels is constant, however the 
rate of over-sampling 66 of pixels is reduced from bottom to top according to a 
scale factor. For instance, if the view at the top of a sample picture is zoomed to 
2X of that of the bottom of the scene 61, the bottom line is over-sampled at a rate 
of 2, while the top line is not over-sampled at all. 
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[0036] Each successive line from bottom to top is over-sampled by the 
equation 68: 

S = l + (Zt-Zb)(N-n)/N 
Where S is the rate of over-sampling, N is the number of horizontal lines of 
pixels, n is the horizontal line of pixels number counting from bottom to top, Zt 
is the zoom ratio at the top of the sample picture, and Zb is the zoom ratio at the 
bottom of the scene 61. A similar equation is applied in the vertical dimension 
results images with correct aspect ratios. Applying Equation 68 in the vertical 
direction results in skipping lines. 

[0037] The invention illustrated in the various embodiments can be used in 
any situation where maximum resolution is required over a large viewing area, 
and the resulting picture distortion is acceptable. It is expected that all 
Government security surveillance will ultimately be performed using digital 
cameras interconnected via digital networks. Most existing surveillance systems 
suffer from reduced resolution in the far field. As such, most government 
security surveillance systems can benefit from this application. Other uses of the 
invention illustrated in the various embodiments include face recognition, 
biojective microscopes, machine vision, and any other use wherein object 
recognition in the far field of view of the picture image would be useful. 

[0038] While the invention may be susceptible to various modifications and 
alternative forms, specific embodiments have been shown by way of example in 
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the drawings and have been described in detail herein. However, it should be 
understood that the invention is not intended to be limited to the particular 
forms disclosed. Rather, the invention is to cover all modifications, equivalents, 
and alternatives falling within the spirit and scope of the invention as defined by 
the following appended claims. 
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